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A compact differentiation technique (without 
using indexes) is developed for scalar functions 
that depend on complex matrix arguments which are 
combined by operations of complex conjugation, 
transposition, addition, multiplication, matrix 
inversion and taking the direct product. The 
differentiation apparatus is developed in order 
to simplify the solution of extremum problems of 
scalar functions of matrix arguments. 


Often in solving the problems of mathematical statistics , /3 * 

the theory of probability, system theory and others it becomes 
necessary to calculate the derivatives of functions that depend 
on matrix arguments. If the relation to the arguments is 
expressed by a complex function, the calculation of the deri- 
vatives by development of the matrix operations in indexes 
becomes extremely laborious. In [1] a number of "recipes" are 
proposed that heighten the "probability" of obtaining a finite 
formula in the matrix expression. One of the authors of the 
present work has previously pointed out a general method of 
solving the problem without recourse to indexes [2] . The 
present work solves this problem in its entirety. 


1. The General Relations of Matrix Differentiation 


The basis of preserving the matrix form of the operation 
of differentiation with respect to a matrix argument is the 
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special form of notation of the derivatives. Let f (x) be a 
scalar function of the matrix argument x; x is a matrix of 
size m X n. We shall introduce the following designation: 
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( 1 ) 


We shall define the operator form of notation for the dif- /4 
ferentiation of complex functions as follows. Let y (x) be a 
matrix that depends on the matrix x, f (y) be a scalar function 
of the matrix argument. By the rule of differentiation of 
complex functions we have : 

1) ±(ycx)) y 

~ if ?X»e <2) 


We shall rewrite relation (2) in the operator form: 

9/ rlM] if 

$x~ LVxJV!/ > 


(3) 


where 3f/3x and 3f/3y are the matrix forms of notation of the 
partial derivatives as defined in definition (1) ; the expression 
in brackets here and in future shall denote an operator which 
by definition operates on the matrix to the right (in the pre- 
sent case, 3f/3y) . If the complex function has the form f(z), 
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where z « 0 (y) , y - y (x) , z,y,x being matrices/ then analogous 
with (2) we have: 

Z£fz) _ j T ?/<# 

9Xmg~ *y** (4) 


or, writing (4) analogous to (3) we obtain in the operator 
form: 


jx~bxJl vyjvz 

In the general case, if the complex function has the form: 

(• * • (Zp))) 

then the following formula is obviously valid: 



Formula (RO) reduces the initial problem to one of operation 
by operators : 



We primarily observe the obvious relation: 

_ r $ x 1 <Ri ^ 
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The following relations enable a notation of the deriva- 
tive in matrix form if the argument of function f is obtained 
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from matrices by the operations of complex conjugation, trans- 
position, matrix inversion (hereafter we write F(y) = 3f(y)/3y) 



(R2) 


(R3) 





(R4) 


The proofs of relations (R2)-(R4) are done in indexes. As 
an example we carry out the proof of (R4) . By definition of an 
inverse matrix: 



( 6 ) 


which is the Kronecker symbol. We differentiate (6) with 

respect to x : 
c me 
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In the second term of (7) we can sum over j. Then we multiply 
(7) by: 


Lx\ 


and sum over all k: 
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Taking account of (6) and summing over k in the second term, 
we obtain from (8) : 


- (x -y 

QXh<e . L * J> - 


(x 1 ! 


m 


•ep 


We shall now prove (R4) in the index form: 

9X.1 i f •>{f% n.e if “ ®V 


(9i 


Taking note of the form of notation (1) , we obtain (R4) in an /6 

obvious manner from (9) in the matrix form. The proof is com- 
plete. 

The next two relations, easily proved in indexes, allow us 
to reduce the calculation of the derivatives of complex func- 
tions to more simple expressions and, together with relations 
(R0)~(R4), allow a calculation in matrix form of the derivatives 
of any given matrix expressions. Let A(x) and B (x) be 
matrices that are functions of the matrix argument x; then: 
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As an example of the use of formulas (R0)-(R6) we shall 
calculate the derivative: 


&rf(CUx r Jxn . 

We denote: y __ 


Step 1. Per (RO) : 
Step 2. Per 


2£ = r2lj 

vx L vx. 





<F~l$xlf Fi 'L'BX. 

where for brevity in writing the further calculations we 
denote : 


Step 4. Per 
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Step 5. Per (R3) : 


Step 6 . Per (R5) : 




Finally, we get: 

—ff'ffr'J T x ^ T -f i / r &f*x T J= 
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= -(X^xjVF^J^X^-^xyh^Jx) X T J. 
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If the reader still has doubts about the usefulness of 
the matrix method of differentiation, it is worthwhile to try 
to obtain formula (10) by calculations with indexes. 

We shall fill out the list of permissible matrix operations 
with the direct product of matrices and derive the corresponding 
differentiation formulas. The direct product of A and B in the 
notation : 


is defined as the block matrix: 




B J /z 8 ... 
J) zz 8 ' • ‘ 
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If matrices A and B are of respective size k X m and r X s, 
according to definition (11) : 




Let A and B be matrices that depend on the matrix x , (_8 

f(ASB) be a scalar function of the matrix A8B; by definition: 
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We obtain an analog of formula (R5) for the case of the direct 
product : 


Z g^c- ,J r+p VXnt, * 

VXa* ype * r r*nj, 

5- F fiM-B v 4 &*** (12) 

< VX* f ^ VXh? J 

* % 


Designating in (12) : 

( 6 ©rtij ~ Je Bf,e > (X3 ) 

(FQjJpe = Z o -/j7+/> ~>?ey . (14) 


By its meaning, the operation © is an operation that is inverse 
to the taking of the direct product. The following formula 
obtains in designations (13) and (14) : 

2. Differentiation of the Trace and the Determinant 

The most common functions of a matrix argument are the 
type tr(x) and det(x) (defined only for the square matrices x) , 
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which it is worth giving a special consideration. In accordance 
with definition (1) : 

9 t*.(xl T 

where I is the unit matrix. In particular, in formula (10) if /9 

f is the trace, P = I, 

Using relations (R4) and (R5) we obtain the formula: 
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which is valid for n = 0, ±1, ±2, .... 


(T2) 


Let P (x) be a matrix function of a matrix argument, which 
can be represented in the form of the Taylor series : 

FM iK xl 

Then, using (T2) , we determine the matrix derivative: 
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In particular: 


^ x 
QX c * 


(T4) 


To obtain the formulas for the derivatives of a deter- 
minant we observe that the determinant can be represented as 
the sum of any given row [3] : 
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where : 




is the algebraic complement of the element and, consequently, 
does not depend on x. . . Thus : 

J 






~X-H Xzy . . . y„ 

^/2 -^22 1 1 ‘ ^ K 2. 
• • • * • • 
.X 4n -X-zh * i • 36i n 


If det(x) ^ 0, it is known that a matrix comprised of algebraic 
complements is expressed in terms of the inverse matrix [3] , 
and therefore when det(x) ? l - 0: 


x'JeiOd. 
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We shall compute the derivative of det(x n ): 


9 JeHxH) - . 9(Jet(x)) n 
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We derive a formula analogous to (T3) for the case of the 
determinant : 
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In deriving (D3) we made use of relations (R6) and (D2) and the 
fact that matrices F(x) and x commute. In particular: 



(D4) 
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